A networked data storage system may include one or more network storage servers, which may be storage appliances. A network storage server may provide services related to the organization of data on storage devices, such as disks. Some of these network storage servers are commonly referred to as filers or file servers. An example of such a storage server is any of the Filer products made by Network Appliance, Inc. in Sunnyvale, Calif. The storage appliance may be implemented with a special-purpose computer or a general-purpose computer. Depending on the application, various networked data storage systems may include different numbers of storage servers.
For financial institutions in the United States, such as security broker firms, security dealers, etc., the Security and Exchange Commission (SEC) imposes additional requirements on the data storage systems of these institutions for regulation and rule enforcement purposes. One of these rules is in 17 C.F.R. § 240.17a-4(f) adopted on Feb. 12, 1997, hereinafter, the “Rule.” The Rule imposes certain criteria on electronic storage media used in the securities broker-dealer industry. Under the Rule, a storage system has to provide non-rewriteability and non-eraseability, in addition to other requirements. When the Rule was developed, the prevalent storage media used in the securities broker-dealer industry was write once read many (WORM) optical disks. The WORM optical disks generally allow immediate verification of the accuracy of the recording process as an integral part of writing a record to the optical disks. Typically, a WORM optical disk immediately reads back the written sectors related to the record in a second revolution of the optical disk and compares what is read with what was intended to be written. If an error is detected, the recording process is retried in another location on the disk.
However, as data storage technology advances, magnetic storage devices, such as magnetic disks, have become the preferred storage devices over optical disks. Advantages of magnetic storage devices include their generally higher capacity and lower cost than optical storage. Another advantage is the shorter time needed to retrieve data from the magnetic storage devices, because optical disks generally take a relatively long time to load (e.g., from an optical silo) before they can be searched. Magnetic storage devices also provide fast random access. Since a regulatory agency may allow a financial institution only a very short time period to respond to a discovery request, it is important to have fast random access to the stored data. Moreover, it is relatively easy and economical to replicate the data stored in a magnetic storage device and to store the copy in another location using replication solutions, such as SnapMirror® from Network Appliance, Inc. The ease in replication facilitates disaster recovery of the data storage system.
Because of the various advantages of magnetic storage devices, it is desirable to use magnetic storage devices in the data storage systems in the securities broker-dealer industry as well. Techniques have been developed to make magnetic storage devices comply with the rules promulgated by SEC. In particular, a WORM storage solution applicable to magnetic storage devices has been developed by Network Appliance, Inc. The WORM storage solution allows for verification of accuracy at the time each record is read and periodically during the storage life of the record. However, since the data written is not verified until the record is read, there is some latency in discovering a problem with the data written. Furthermore, the periodic verification capability essentially transfers the responsibility for the timing of verifying the accuracy with which the recording was done to the system administrator, who then chooses the frequency and the immediacy with which the verification is performed. In addition to burdening the system administrator with the responsibility, the reliability of the storage system may be compromised if the frequency of verification is set too low. On the other hand, the performance of the system may suffer if the frequency of the verification is set too high.